Methods for deciding what to do next and learning*

Recent years has seen intense analysis and questioning of the importance of the role of classical AI-planning in deciding moment-to-moment actions. This has led to development of several new AI-planning paradigms such as reactive planning [Fir87, GL87]. Informally, a planning paradigm refers to principles ofrepresenting an agent's acts, beliefs and expectations about concepts-relatingacts-to-conditions-in-the-world, and methods for prescribing acts in response to conditions. Each paradigm (independent of a domain) develops a uniform way of interacting with the world and as such it attempts to account for complexities in the agent's environment. The oldest AI-planning paradigm, known as classical planning, is closely related to reasoning and as such it is a highly cognitive behavior involving explicit goals. On the other hand, the reactive planning paradigms use little or no reasoning and goals are implicit. Reactive planning clearly has a less cognitive character than classical planning. We are interested in agents that both act based on improvised reactions, producing whatever goals are implicit in those improvised reactions, and generate plans to achieve explicit goals. We will refer to these ways of behaving as the agent's modes of behavior. By what to do next we have in mind the very next physical action an agent situated in the world performs. Actions we consider are provoked either by direct sensing and a specific purpose (i.e., a goal) or by direct sensing and a general condition (i.e., an implicit goal). "Methods for deciding what to do next" in the title of our paper is meant to cover all planning paradigms that advance a physical action for execution, subject to a few simplifying assumptions. This includes actions prescribed by a plan produced by a classical planner, actions suggested by a reactive planner, and actions suggested by experimenting either to gain information about an agent's capabilities or to achieve a goal heuristically. Since the scope of planning methods differ about when and in what ways the agent may use the output, we suggest the following assumptions on planning methods to limit their scope for deciding what to do next. These assumptions do not affect how the methods work. They only affect the inputs given to the method and application of their output.